Lossy Computing of Correlated Sources with 

Fractional Sampling 

Xi Liu, Student Member, IEEE, Osvaldo Simeone, Member, IEEE, and Elza Erkip, Fellow, IEEE 



O 
(N 

> 
O 

m 
(N 



(N 
> 

00 

m 
o 



X 



Abstract — This paper considers tlie problem of lossy compres- 
sion for the computation of a function of two correlated sources, 
both of which are observed at the encoder. Due to presence 
of observation costs, the encoder is allowed to observe only 
subsets of the samples from both sources, with a fraction of such 
sample pairs possibly overlapping. The rate-distortion function 
is characterized for memoryless sources, and then specialized 
to Gaussian and binary sources for selected functions and with 
quadratic and Hamming distortion metrics, respectively. The 
optimal measurement overlap fraction is shown to depend on 
the function to be computed by the decoder, on the source 
statistics, including the correlation, and on the link rate. Special 
cases are discussed in which the optimal overlap fraction is the 
maximum or minimum possible value given the sampling budget, 
illustrating non-trivial performance trade-offs in the design of the 
sampling strategy. The trade-off between the average distortion 
and the worst-case distortion is then studied. Finally, the analysis 
is extended to the multi-hop set-up with jointly Gaussian sources, 
where each encoder can observe only one of the sources. 



I. Introduction 

Consider an encoder endowed with a sensor that is able to 
measure two correlated discrete memoryless source sequences 
5'" = (S'i,i,...,5i,„) and = {82,1, S2,n), as shown in 
Fig.[T] Due to the energy cost of source acquisition, sampling, 
quantization and compression, it might not be possible for the 
sensor to fully measure the sources 5*1 and 82- To simplify, 
this limitation can be modelled by imposing that only ndk 
samples can be measured from each source Sk, k = 1,2, with 
< (^k < 1- The encoder compresses the measured samples to 
nR bits, where R is the communication rate in bits per source 
sample. Based on the received bits, the decoder reconstructs a 
lossy version of a target function T" = /"(S*", S'2 ) of source 
sequences 5" and S'2 , which is calculated symbol-by-symbol 
as Ti — f{Si^i, 82,1), i — l,...,ri. We refer to the above 
problem as lossy computing with fractional sampling. 



Fig. 1. The encoder measures correlated sources 5i and S'2 for a fraction 
of time 61 and 82, respectively, and the decoder estimates a function T" = 

A key aspect of the problem of lossy computing with 
fractional sampling is that the encoder is allowed to choose 
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Fig. 2. The multi-hop setup studied in Section VII: Encoder 1 and Encoder 
2 measure correlated sources Si and S'2 for a fraction of time 9i and 82, 
respectively, and the decoder estimates a function T" = f^{S" , SJ). 



which samples to measure given the sampling budget {61, 62)- 
To fix the ideas, assume that we have (01,62) — (0.5,0.5), 
so that only half of the samples can be observed from both 
sources. As two extreme strategies, the encoder can either 
measure the same samples from both sources, say Si,i,S2,i 
for i = l,...,n/2, or it can measure the first source S'l 
for the first n/2 samples, namely Si,i for i ~ l,...,n/2, 
and the second source S'2 for the remaining n/2 samples, 
namely 82,1 for i = n/2 + l,...,n. With the first sampling 
strategy, the encoder is able to directly calculate the desired 
function Ti — f{Si^i, 82,1) for i = l,...,n/2, while having 
no information (beside the prior distribution) about Ti for 
the remaining samples. With the second strategy, instead, the 
encoder collects partial information about T at all times in the 
form of samples from source Si or source 82- 

Relating the discussed fractional sampling model with prior 
literature, we observe that, with full sampling of both sources, 
i.e., (6*1 — 1,62 = 1), the encoder can directly calculate the 
function T" — /"(S'", S2) and the problem at hand reduces to 
the standard rate-distortion set-up (see, e.g., lH]). Instead, if the 
encoder can only measure one of the two sources, i.e., (6*1 = 
1, 02 = 0) or (9i — 0,02 — 1), the problem at hand becomes a 
special case of the indirect source coding set-up introduced in 
||2l. The framework of source coding with fractional sampling 
was introduced in our previous work [3 1 for a model in which 
an energy-constrained sensor measures independent Gaussian 
sources and the receiver wishes to reconstruct all sources with 
given quadratic distortion constraints. The model of fractional 
sampling is also related to that of compression with actions of 
[4 1, in which the decoder obtains side information by taking 
cost-constrained actions based on the message received from 
the encoder 

The main contributions and the organization of the paper 
are as follows. We formulate the problem of lossy computing 
with fractional sampling of correlated sources for the set-up in 
Fig. [T] in Section II. After providing a general expression for 
the distortion-rate and the rate-distortion functions in Section 
III, we specialize it to Gaussian sources and weighted sum 
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function T = wiSi + W2S2 in Section IV and binary sources 
with arbitrary functions T — f {81,82) in Section V. As 
a result, various conclusions are drawn regarding conditions 
under which the optimal sampling strategy prescribes the 
maximum or the minimum possible overlap between the 
samples measured from the two sources. Next, we investigate 
the trade-off between the average distortion and the worst- 
case distortion in Section VI. In Section VII, we extend the 
analysis to the multi-hop set-up of Fig. |2] in which sources 
81 and 52 are measured by different encoders connected by a 
finite-capacity link. 

II. System Model 

In this section, we formally introduce the system model of 
interest for the point-to-point set-up of Fig. [T] The multi-hop 
model will be introduced in Section VII. As shown in Fig. 
[1] the encoder has access to two discrete memoryless source 
sequences 5" = 5i,„) and S'J = (5*2,1, 6*2,, i) 

respectively, which consist of n independent and identically 
distributed (i.i.d.) samples (5i,i,52,i) with 5i,,; G Si and 
52. i G 52, * = 1, n, where Si and ^2 are the alphabet sets 
for 5i and 52 respectively. All alphabets are assumed to be 
finite unless otherwise stated. Due to presence of observation 
costs, we assume the encoder can only sample a fraction Ok 
of the samples for source 8k, with < 9k < I for k = 1,2, 
where the samples are determined prior to the observation of 
(5", 52). Given the i.i.d. nature of the sources, without loss 
of generality, we assume that the encoder measures the first di 
fraction of samples of source 5i and measures the 62 fraction 
of samples of 52 starting from sample n{9i — 612) + iQ, as 
shown in Fig.|3] The samples measured at the encoder from the 
two sources thus overlap for a fraction 6*12, with 612 satisfying 
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with 9i2,,nin = {01 +02 - 1)+ and 0i2,rnax = min(6'i , 6*2 ), 
where (•)+ denotes max(-,0). We refer to the triple 
{61,62,012) as a sampling profile, and to {61,62) as the 
sampling budget. The decoder wishes to estimate a function 



0(1+6!-, -6! -ft) 



Fig. 3. Sampling profile (Oi, 62,012) at the encoder: a fraction, 81 — 812, 
of samples is measured only from source Si; a fraction, 612, of samples is 
measured from both sources; a fraction, 82 — 612, of samples is measured 
only from source S2; and the remaining fraction, 1 + 612 — 61 —82, of 
samples is not measured for either source (0 < ^li^2 < 1, and 612 as in 

O). 

T" =. r{8^,8l), where T, = /(5i,„52,.) for ^ = l,...,n. 
We let d : T x T — > [0, +00) be a distortion measure, where 
T and T are the alphabet sets of the variables T and T 
respectively. We assume, without loss of generality, that for 



each t ^ T there exists a i E T such that d{t, t) — 0. The 
link between the encoder and the decoder can support a rate 
of R bits/sample. Formal definitions follow. 

Definition 1: A {n, R, 0,61,62,612) code for the problem 
of lossy computing of two memoryless sources with fractional 
sampling consists of an encoder h : 5"^^ x 5. 
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{1, ...,2"^}, which maps the measured 0i-fraction of source 
5i, i.e., (5i,i, 5i,„ej), and the measured 02-fi"action of 

source 52, i.e., (52,„(ei-ei2)+i, 82,n{ei+92-9i2))' 1"^° ^ 
message of rate R bits per source sample (where the normal- 
ization is with respect to the overall number of samples n); and 
a decoder g : {!,..., 2"^} T", which maps the message 
from the encoder into an estimate T", such that the average 
distortion constraint D is satisfied, i.e.. 



-E 



.i=l 



d{T„Ti, 



< D. 



(2) 



Given any sampling profile {61,62,612), a tuple 
{R, 0,61,62, 612) is said to be achievable, if for any e > 0, 
and sufficiently large n, there exists a (n, R,D + e,6i,62, 612) 
code. The distortion-rate function for a given sampling profile 
D{R,6i,62,6i2) is defined as D{R,6i,62,6i2) = inf{L>: 
{R, D, 61,62, 612) is achievable}. The distortion- rate function 
with sampling budget {61,62), D{R, 61,62), is defined as 
D{R,6i,62) — mingjj D{R,6i,d2,6i2) where the minimum 
is taken over all 612 satisfying ([T). Similar definitions are used 
for the rate-distortion function. Specifically, the rate-distortion 
function given a sampling profile {61,62,612) and distortion 
D is defined as R{D, 61,62, 612) = inf{i?: {R, D, 61,62,612) 
is achievable}, and the rate-distortion function with sampling 
budget (6*1, 6*2) as R{D, 61,62) = min^,, i?(i:>, 6*1, ^2, ^12) 
where the minimum is taken over all 6*12 satisfying ([T]i. We 
note that, while (|2j measures the average expected distortion 
across all samples, in Section VI we will consider also the 
worst-case expected distortion maxjgji^ „-ji?((i(Ti,Ti)), 
where the expectation is taken over the source distribution. 

III. Rate-Distortion Trade-Off with Fractional 
Sampling 

In this section, we characterize the distortion-rate functions 
D{R,6i,62,6i2) and D{R,6i,62) defined above as well as 
their rate-distortion counterparts. To elaborate, we first define 
the standard distortion-rate function for the memoryless source 
T as Di2{R) = ^^^.p^i\ty. iiT-j^)<RE[d{T,f )] QJ. We simi- 
larly define the corresponding rate-distortion function with full 
sampling as Ri2{D) = ^.m^^^^^^y Eld{T,f)]<D HT;f). More- 
over, we define the indirect distortion-rate function for com- 
pression of source T when only 8k is observed at the encodeJl, 
for k = 1,2, as Dk{R) = min^^^^^^y^ ^^g^.^^^^j,E[d{T,f)]. 
We similarly define the corresponding rate-distortion func- 
tion Rk{D) = niinj,(i-|,^)^£;[rf(j,t)]<D^('5fc;^)- Finally, 
we define Dk,rnin as Dk,min = oc Dk{R) = 

mmgi^(^.'j E{d{T, gk{8k))), for fc = 1,2, where function gk{-) 
is a function gk : <Sfc — s> T. 



'Throughout the paper, quantities such as n6i, n92 and n{8i + 62 - 
are implicitly assumed to be rounded to the largest smaller integer. 
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^In this case, for any k, k' G {1, 2} and k k',f ■ 
a Markov chain. 



5j; — > Sfe/ forms 



3 



Lemma 1: For any given sampling profile (6*1, 02, ^'12) and 
link rate R, the distortion-rate function for computing T is 
given b}0 

Ri 



D{RA, 02,0,2)= mill (01 -012) A 

-Rl ,fll2,-R2>0 \ Cl — 1712 



i?12 



712^12 {^]+{02- Oi2)D2 

'12 J \^2— C12 



i?2 



(1 + 012 - 01 - 02)A, 



(3) 



with D„iax = niin^g^ £'[(i(T, t)], and where the minimization 
is taken under the constraint Ri + R2 + i?i2 < R- Moreover, 
defining 

Dmin{0i,02,0i2) = lilTL 0i , 02 , 012) 

= (01 — 012)-Dl,.mm + (02 — 012)^*2, mm 
+ (1+012- 01 -02)i^™ax, (4) 

for any given sampling profile (0i,02,0i2) and distortion 
level D > £'mm(0i, 02, 012), the rate-distortion function for 
computing T is given by 



RiD A, 02,0,2) = ^ mill (01 -0i2)i?i 



+ O12R12 



012)^2 



D2 



(5) 



'12 / \ C2 — P12 , 

where the minimization is taken over all choices of D,, D2 and 
D12 satisfying D12 > 0, Dk > (0fc - 0i2)£»fe,m««, k = 1,2, 
and 

D1+D2+ D12 + (1 + 012 - 01 - 02)^ma. < D. (6) 



In the lemma above, rate Rk is assigned for the description 
of the (0fe — 0i2)-fraction of samples in which only source 
Sk is measured, k — 1,2, while rate R12 is assigned for 
the description of the 0i2-fraction of samples in which both 
sources are measured (recall Fig. |3). Distortions D,, D2 and 
D12 are the weighted average per-sample distortions in the 
reconstruction of T at the decoder for the corresponding 
fractions of samples. The proof follows immediately from 
the independence of the samples measured from the different 
fractions, and it is thus omitted. We will need the following 
two lemmas. 

Lemma 2: For any given sampling budget (0i,02), with 
i? — > 00, the minimum achievable distortion for computing 

T = /(5i,52)is given by 

Dmin{0i,02) =(01 — 0*2)0, jnin + (^2 ^ ^12)^2, mm 

+ (1 + 0*2-01 -02)i?ma., (7) 

where 0^2 = 012, max if D^ax < Di jnin + D2^min, ^12 = 

6'i2,mi« if D,nax > ^'i.min + -D2,™i„, and 0^2 is arbitrary if 

-^max — Ol,min + -^2,mm- 

Proof: It follows by considering the monotonicity of (|4]l 
with respect to 0i2. ■ 

^For the distortion function Di{x) for x > 0, we define ■ Di{x/0) = 
0, for a; > 0, if limj^-jo x • Di{l/x) = 0, and similarly for the distortion 
functions Di2{x) and D2{x). 



Lemma 3: D{R, 9,, 62) is continuous and convex in R for 
R> 0. Similarly, R{D, 9,, 02) is continuous and convex in D 
for D>D„,„,{9,,92). 

Proof: It follows from the operation definitions given in 
Definition 1 similar to |1, Thm. 10.2.1]. ■ 

IV. Gaussian Sources 

In this section, we focus on the case in which sources 
Si and 5*2 are jointly Gaussian, zero-mean, unit-variance 
and correlated with coefficient p, with p G [—1,1]. The 
decoder wishes to compute a weighted sum function T = 
wiSi + W2S2, with wi, W2 S K, under the mean square error 
(MSB) distortion measure d{t, t) = {t — t)^. In the following, 
we study two specific choices for the weights w, — l,W2 = 
and wi = W2 = 1, resulting in the functions T — S, and 
T = 5i + S'2, respectively. These two cases are selected in 
order to illustrate the impact of the choice of the function 
/(S'1,5'2) on the optimal sampling strategy. The discussion 
can be extended with appropriate modifications to arbitrary 
choices of weights {wi,W2)- 



A. Computation of T = Si 

Proposition 1: For a given sampling budget (0i, 
distortion-rate function for computing T = 5i is 



D{R,9,,92) 



1-9, +9,2' 
1-01 -p2(0 



-0 



= < 



'12 



ifR<f log: 

12) 

2" 



02), the 



(p2^ 81+62-0^2 ^ Otherwise 



(8) 

where 0*2 = 0i2,min is the optimal overlap fraction. The rate- 
distortion function i?(_D,0i,02) can be obtained by inverting 
function (O with respect to variable D. 

Proof: See Appendix lAl ■ 
Proposition [T] confirms the intuition that if the receiver 
is interested in source 1 only, i.e., T — S,, the encoder 
should simultaneously measure both sources S, and S2 for 
a fraction of time to be kept as small as possible. Moreover, 
if i? < 0i/21og2(l//5^), the entire rate R is used to describe 
only the 0i -fraction of samples measured from source Si only; 
otherwise, both the 0i-fraction of source S, and the (02 — 012)- 
fraction of source ^2 that is not overlapped are described at 
positive rates. 

B. Computation of T — S, + S2 

We now consider the case in which the desired function is 
T = 5*1 + ^2. Note that T is a Gaussian random variable with 
zero mean and variance D„iax = 2(1 + p), and that T and 
Si (or 5*2) are jointly Gaussian with correlation coefficient 
p = •\/(l + p)/2. Moreover, since T ~ for p = — 1, it is 
enough to focus on the interval p G (—1,1]. Finally, we recall 
that the distortion-rate function for T is given by Di2{R) = 
2(1 + p)2"^^ for R > [IJ, and the indirect distortion-rate 
function is DkiR) = 2(1 + p){l - p"^ + p^2-'^^) for i? > 
and fc = 1,2 0. 
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Proposition 2: Given sampling budget (6*1,02), the 
distortion-rate function for computing T = Si + S2 i& 



min (1 + pY 



DiR,9i,02) 

+ 2(l + p) [1 + pdu + 0122' 



02 - 26'i2)2"''i+''2-2«i2 



(1 + Pf 



h+02), 

(9) 

where the minimization is taken over all 0i2 satisfying ^ and 
all R12 satisfying R12 < R. 

Proof: This proposition follows by Lemma [T| using argu- 
ments similar to the ones in Appendix |A] ■ 

In order to obtain further analytical insight into (|9) and the 
optimal sampling strategy, we now consider some special cases 
of interest. 

Proposition 3: For i? ^> 00, we have 

Drmniei,02) = 2(1+ p)il+p9l2)-{l+pf {01+02), (10) 

where 0^2 = 0i2,min if p > 0, 0^2 = 0i2,rnax if p < 0, and 
is arbitrary if p = 0. 

This proposition is easily obtained from Lemma |2] It says 
that, if the sources (6*1,52) have positive correlation, i.e., 
p > 0, and there are no rate limitations (R cxo), the MSB 
distortion increases linearly with 6*12, and it is thus optimal 
to set 012 to be the smallest possible value 6*^2 = 0i2.min- 
In contrast, if p < 0, the MSB distortion decreases linearly 
with 612, and thus the optimal 0^2 is the largest possible 
value, dl2 = 0i2.7nax- This shows the relevance of the source 
correlation in designing the optimal sampling strategy. 

The general conclusions about the optimal sampling strate- 
gies discussed above for infinite rate can be extended to finite 
rates R in certain regimes. Specifically, Proposition |4] below 
states that if p < 0, then, just as in the case of i? -> cx) 
of Proposition [3] the encoder should set 6*12 to be as large 
as possible, i.e., 6*5^2 — 6*12. maa;, irrespective of the value 
of R. Furthermore, Proposition |5] below demonstrates that, 
for sufficiently small rates, the optimal overlap 6**2 tends 
to be maximum, i.e., ^12 = 0i2.max, for a larger range of 
correlation coefficients p than p < 0. This is mainly because 
when rate R is small enough, it is generally more efficient 
to use the available rate to describe T directly during the 
overlapping 0i2-fraction, rather than indirectly describing T 
based on observations of Si or S2 alone. 

Proposition 4: For p < 0, the distortion-rate function is 



D{R,9i,02 



where 



2 



"12 

»l+<'2-»I 



2(1 



2) 



-{l + p)H9 



2(l + p) 1 



p)(l + p0 

2), if i? > %log2 

, otherwise, 
(11) 



01*9 2 



2R 



2 
1+P 



^12 = 9i2.max is the Optimal overlapping fraction. 
Proof: The proof is obtained by solving (|9]l for p < 0. ■ 

Proposition 5: For any p > 0, if i? < 
(6*12, mm/2) log2(2/(l + p)), the distortion-rate function 



IS given as 



— p = -0.5 

-■p = 

- - p = 0.5 



D{R, 9i,02) = 2(1 + p)(l - 0I2) + 2(1 + p)0l22 , (12) 



2R 



Fig. 4. Distortion-rate function for computing T = Si + S2, {81,82) 
jointly Gaussian with correlation coefficient p = —0.5, 0, 0.5, respectively. 
Tlie sampling budget is (81,62) = (0.5,0.75). 



where 0I2 = 0i2,max- 

Proof: Given R < (6li2^™„/2) log2(2/(l + p)), for 
any feasible 6*12 satisfying ([T]l, we always have R < 
(6*12/2) log2(2/(l + p)). In this case, for any given 6*12, 
applying the standard Lagrangian method to (|9}, we obtain 
RI2 = R- Substituting into Q and considering the mono- 
tonicity of function D{R,0i,92,0i2) with respect to 6*12, 
we can show that the optimal overlap fraction is given by 
6*12 = 0i2,7nax, leading to the distortion-rate function as stated 
in the proposition. ■ 

C. Numerical Results 

In this subsection, we numerically evaluate the distortion- 
rate function for computation of function T = Si + S2- Recall 
that for T = Si, the optimal overlap fraction is always 0i2,min- 
Fig. |4] and Fig. |5] show the minimum MSB distortion D and 
the optimal overlap fraction 0^2 versus rate R, respectively, 
for (6*1, 6*2) = (0.5,0.75), and p = -0.5,0,0.5. The curves 
are obtained by numerically solving the optimization in (|9]l. 
It can be seen from Fig. |5] that, as predicted by Proposition 
m the optimal overlap fraction 0^2 is equal to the maximum 
possible fraction 0i2.max = 0.5, for p = —0.5 < and p = 0. 
Moreover, for p = 0.5 > 0, with sufficiently small rates R, 
as described in Proposition |5] the optimal overlap fraction 0^2 
equals to the maximum overlap 0i2.max — 0.5. However, as 
R increases, 0^2 drops to the minimum value 0i2.min = 0.25, 
which is consistent with Proposition |3] 

V. Binary Sources 

' In this section, we consider binary sources so that iSi = 
S2 = T = T = {0, 1}, and {Si, S2) is a doubly symmetric 
binary source (DSBS) characterized by probability Pr[S'i / 
5*2] — p, < p < 1/2. We take the Hamming distortion as the 
distortion measure, i.e., d{t, t) — l — 5^i, where — \\ft — t 
and (5(£ = otherwise. Since all non-trivial binary functions 
are equivalent, up to relabeling, to either the exclusive OR or 
the AND |6|, it suffices to consider only these two options 
for function T = / (5*1, 5*2): (0 the exclusive OR or binary 
sum, i.e., T — Si® S2', (H) the AND or binary product, i.e.. 
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p . -0.5 

p.O 

■ p. 0.5 



Fig. 5. Optimal overlap fraction th^t minimizes the average expected 
distortion as a function of rate R for computing T = Si + S2, {Si,S2) 
jointly Gaussian with correlation coefficient p = —0.5, 0, 0.5, respectively. 
The sampling budget is (6*1,612) = (0.5,0.75). 



T = Si <S> 82- In the following, we focus on deriving the 
rate-distortion ^i, 6*2) for convenience, since in general 

it takes a simpler analytical form as compared to the distortion- 
rate function D{R, 9i, 02)- 

A. Computation of T ^ Si ® S2 

Proposition 6: For given sampling budget 62), the rate- 
distortion function for computing T = © S'2 is given by 

R{D,ei,e-2)^ I if{l-6l^)p<D<p, (13) 

[0, ifD>p, 

where h{x) — — xlog2(a;) — (1 — x) log2(l — x) is the binary 
entropy function, and 6**2 = Oi2.max is the optimal overlap 
fraction, for (I - 6l2)p < D < p. 

The above proposition can be proved by using the fact 
that T = 5*1 © 52 is a Bernoulli(p) random variable, and is 
independent of 5*1 and 5*2. Therefore, the observation of either 
Si or 52 is not useful for computing T, and thus one should 
choose the overlap fraction to be as large as possible, i.e., 
^12 — ^12, max- The rate-distortion function (fT3] l then follows 
immediately from the rate-distortion function of the binary 
random variable T |[T]. 

B. Computation of T = Si ® S2 

In this subsection, we focus on the binary product T = Si® 
S2, which is Bernoulli distributed with probability (1 —p)/2. 
For convenience, we start by finding the minimum possible 
distortion at the decoder given (^1,^2), i-c, £'mm(6'i, ^^2) as 
defined in Lemma [3] and the minimum required rate to achieve 
it. Then, we proceed to derive the rate-distortion function. 

Proposition 7: For given sampling budget (^i, (^2), the min- 
imum achievable distortion for computing T — 5i ® 52 is 
given by 

l-p , f^^ 1\ N VI 

(14) 



D. 



2) = 



P 



where = fi2,mm if P < 1/3 and 6*12 = Oi2^,nax if 1/3 < 
p < 1/2. Moreover, distortion £'mi„(6'i, 6*2) can be achieved 
as long as i? > ^2) = 9i + 02 - {2 - h (i^)) Ol^- 

Proof See Appendix IbI ■ 

The results in Proposition Q can be seen as the counterpart 
of Proposition [3] for binary sources. In fact, they show that, 
for sufficiently large i?, if p < 1/3, the average Hamming 
distortion increases linearly with 612 and thus we should set 
6*12 to the smallest possible value ^12. mm; instead, if 1/3 < 
p < 1/2, the optimal value of 612 is the largest possible, 
namely, 9i2^,nax- 

Before we proceed to investigate the general rate-distortion 
function R{D,9i,92), we first derive the indirect rate- 
distortion function Ri{D) when only Si is observed at the 
encoder 

Lemma 4: The indirect rate-distortion function for T = 
Si (g) 52 is given by 



RiiD) 



1—P — 2D ^ ^ -, 



h^ + y{\-p) 
U(2d + y(l-2p)+p-l) 



p-\ 



-Mv) 



1<D< 



i-p 
2 ' 



0, D> 



(15) 

Proof: See Appendix |C] ■ 
By symmetry, the indirect rate-distortion function R2{D) 
for T when 52 is observed at the encoder is also given 
by Lemma |4] The rate-distortion function Ri2{D) for T is 
obtained from standard results UJ as Ri2{D) — ft,((l— p)/2) — 
h{D) if < £> < (l-p)/2,andi?i2(-D) = Oif £> > (l-p)/2. 

Proposition 8: For a given sampling budget {9i,92), the 
rate-distortion function for computing T — Si S2 is given 
as 



R{D,9i,92) 



mm 

2,n3,Di2 
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92 — 29i2)Ri 
if D 
if D> 



Di 



min {01 1 ' 
1-p 



1 -\- 02—20 12 



(16) 



where Dmin{0i,92) is as given in Proposition [T] and the 
minimization is taken over all choices of 9 12, D3 and D12 
such that O is satisfied, p{0i + 02 - 26li2)/2 < D3 < 
{l-p){9i + 92~29i2)/2, p9i2/2<Di2 < {l-p)0i2/2, and 



D3 + D12 + (^^-j^^ (1 
Proof: See Appendix ID] 
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= D. 



(17) 



C. Numerical Results 

In this subsection, we numerically evaluate the distortion- 
rate function for computation of function T — 5i (8) 52. 
Recall that for T = 5i © 52, the optimal overlap fraction is 
always 0i2,max- Fig- |6]and Fig. [7] plot the minimum average 
Hamming distortion D and the optimal overlap fraction 0I2 
for (6*1, 6*2) = (0.5,0.75), and p = 0.1,0.2,0.4. In Fig. i] as 
predicted by Proposition [T] the minimum rate Rmin{0i,02) 
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Fig. 6. Distortion-rate function for computing T = Si^S2, {81,82) doubly 
symmetric binary with probability Pr[5i 7^ 52] equal to p = 0.1,0.2,0.4, 
respectively. The sampling budget is (61,62) = (0.5,0.75). 

that achieves distortion Dmin{di,d2), is given by 0.9982, 
0.9927, 0.69 for p = 0.1,0.2,0.4, respectively. It can be 
observed from Fig. |7] for p = 0.4 > 1/3, the optimal 
overlap fraction is equal to the maximum possible value 
^12, max = 0.5, for any < i? < 1. However, for smaller 
probabilities p = 0.1,0.2, the optimal overlap fraction equals 
to the maximum possible value Ou^max = 0.5 for sufficiently 
smaller rates and then drops to the minimum possible value 
Oi2,min — 0.25 once R gets larger Moreover, the smaller 
the probability p is, the larger range of rates R over which 
the optimal overlap fraction 6**2 is 0i2,min = 0.25. We note 
that with a larger p, it is easier to describe T directly, since 
T ^ Bemoulli((l —p)/2), but the indirect description of T 
based on Si or S2 becomes more difficult since T becomes 
less correlated with Si or 5*20 This explains why the optimal 
overlap fraction should be chosen as the maximum possible 
value 9i2,max — 0.5 when p is larger than 1/3 (see the 
curve p — 0.4). In this sense, the regime p > 1/3 may be 
considered as the binary counterpart of the regime p < 
for the Gaussian sum case in Section IV-B. For probabilities 
p < 1/3, the numerical results above imply that the optimal 
overlap depends on the link rate R. Similar to the Gaussian 
sum case when p > (Proposition |5]l, when R is sufficiently 
small, it remains optimal to choose the overlap fraction to be 
the maximum possible; however, as R grows sufficiently large, 
it is more advantageous to have the overlap fraction as small 
as possible, which is consistent with Proposition |7] 

VI. Trade-Off between Average Distortion and 
Worst-Case Distortion 

In the problem formulation considered so far, the goal was 
minimizing the average distortion D{R, 6*1, 6*2 ) at the decoder 
for any given rate R and sampling budget (01,6*2). As a 
result of the average of over all the source samples in (|2), 
this performance metric does not make guarantees on the 
maximum average distortion per sample. In fact, as seen in (|3]), 
the average distortion is the average of the distortions accrued 
over the four relevant fractions of samples illustrated in Fig. [3] 



Fig. 7. Optimal overlap fraction 'hat minimizes the average expected 
distortion as a function of R for computing T = Si ® S2, (Si, ^2) doubly 
symmetric binary with probability Pr[5i 7^ S2] equal to p = 0.1,0.2,0.4, 
respectively. The sampling budget is (61,62) = (0.5,0.75). 



namely the fraction of samples in which only 6*1, both 5*1 and 
S2, only 52 or neither Si nor ^2 are measured. This section 
extends the analysis in the previous sections in order to allow 
the decoder to strike the desired balance between the average 
and the worst-case distortions. 



{9i, 62,012) 
we define 



A. Formulation for General Sources 

For any given sampling profile 
and any rate allocation i?2, ^12) 

Dtju{Ri, R2, Ri2,0i, 02,612) as the maximum average 
distortion among all the sampling fractions shown in Fig. 2, 
i.e., 

Ri 



Dyj{Ri,R2, R12, 01, 62,012) — max 
R2 
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(18) 
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+ 012-Dl2 



mm I 
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l2)Di 
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'*The correlation between T and Si or ^2 is given by ^(1 — p)/(l + p). 



where the indicator function ^{A] takes value 1 if A is true and 
otherwise. We then define the weighted sum of the average 
distortion in (|2]i and the worst-case distortion (fT¥l as 

Ri 

72,^12) 

R12 

+ {1 + 612 - 01 ~ 02)Dmax 

+ ^Dw{Rl, R2, Rl2,Ol,02,Ol2), (19) 

where /i > is the relative weight of the worst-case distor- 
tion. Accordingly, for any given rate R and sampling budget 
{01, 02)7 the modified distortion-rate trade-off is characterized 
by D^{R,Oi,02)= inU,, {R, Oi,02,Oi2)- 

In the case of sampling budget (0\,02) satisfying Oi + 
6*2 < 1, regardless of the choice of the overlap fraction 
6*12 and of the rate allocation {Ri, R2, R12), the worst-case 
distortion occurs during the (1 + 6*12 — Oi — 6*2) -fraction of 
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Fig. 8. Optimal overlap fraction that minimizes distoition Z)p(i?, 61, 62) 
as a function of /i for computing T = S1+S2, {Si, S2) jointly Gaussian with 
correlation coefficient p = 0.5. The sample budget is (Si, 62) = (0.5,0.75) 
and the link rate is i? = 0.3, 0.6, respectively. 



samples in which neither source is measured and therefore we 
have Dw{Ri,R2, R12, 01,02, 812) = D^ax- In this case, the 
addition of a constant term in the objective function of (fT9] l 
does not affect the optimization and thus the optimal overlap 
fraction 0*2 and the optimal rate allocation {Rl, R2, RI2) 
remains the same as in the case when fj, = 0. Therefore, in 
the following, we focus on the nontrivial case where we have 
< 6'i < 1, < 6*2 < 1 and 0i + 62 > 1. In this regime, 
function D^{R, 9i,92, 6*12) takes different forms depending on 
the value of 9i2- Specifically, if the overlap fraction 6*12 is 
selected such that 9i2.min < 9i2 < 6*12, maa;, then we have 

Ri 



D^{R,9i,92,9r2)= min (0i - 0i2)i?i 

Hl,i?.12,fl2>0 \ (71 



712 



9i2D 



12 



?2 — 9i2)D2 



R2 



712 / \02 
(1 + 9i2 — 9i — 92)Dmax + fJ-Drnax- 
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(20) 



This is because, with 6*12, min = + 6*2 — 1, we have 1 + ^12 — 
Oi - 82 > and accordingly D-u}{9i,92,9i2, Ri, R2, R12) = 
Dmax- The minimum distortion (l20l i when 9i2 is in the interval 
9i2,min < 9i2 < 9i2,max thus Can be obtained in the same 
manner as done in the previous sections when fi = 0. On 
the other hand, if the overlap fraction is 9i2 ~ ^^12, mm = 
9i + 82 — 1, then we can write 



D^{R, 8i,92,9i2: 



min {l-e2)DJ-\- 
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,1-01 

The minimization (ISTT l is discussed below for the Gaussian 
case. The minimization of (fT9] l is then obtained by taking the 
minimum between (ISTT i and the value obtained for the interval 

8l2,min ^ 9\2 — 9\2,max 

from (|20li. 



B. Computation of the Sum of Jointly Gaussian Sources 

In the special case of Gaussian sources 6*1 and ^2 and of 
calculation of the function T = 5i + 5*2 as in Section IV-B, 
we can simplify (I2TI 1 as 



D^{R,8i,92,9 

12, mm ) 
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(23) 



where ^} follows from the facts that Di{R) = D2{R) for 
R > and that the function is minimized when i?i/(l — 
82) — R2/{1 — 9i); and (|23] | follows because the constraint 
Ri + R2 + R12 < R is easily seen to be met with equality in 
any optimal solution. 

We now numerically evaluate ( |23] |. the trade-off between the 
average distortion and the worst distortion for the Gaussian 
case at hand. Fig. |8] shows the optimal overlap fraction 9^2 
versus the relative weight /i when (81,82) = (0.5,0.75) and 
p = 0.5, for two choices of rates R = 0.3 and R = 0.6 
respectively. From the curve with R = 0.3, we can see that, 
for sufficiently small /x, here, /i < 0.069, the optimal sampling 
fraction is 9i2,max = 0.5, consistent with the result of Fig. |5] 
However, as ^ grows larger, the optimal sampling fraction 
drops to the minimum possible value 0i2,min = 0.25. This is 
since, as the value of /i grows sufficiently large, one needs 
to keep the worst distortion as small as possible. Similar 
conclusions are reached for R = 0.6, except that the optimal 
sampling fraction for sufficiently small ji {p. < 0.004) is 0.42 
instead, which is also consistent with the result of Fig. |5] 

VII. Multi-Hop Lossy Computing with Fractional 
Sampling 

In this section, we extend the analysis of lossy computing 
with fractional sampling from the point-to-point setup to a 
multi-hop setup as depicted in Fig.|2] We assume that Encoder 
k can only sample a fraction 9^ of the samples for source Sk, 
with < 0fc < 1, for k = 1,2. Moreover, the encoders make 
decisions on which samples to sense independently and based 
only on the statistics of the sources. In particular, to ensure 
causality, Encoder 2 is not allowed to observe the message 
from Encoder 1 before making a decision on which samples 
to measure as instead assumed in Q for a related set-up. 
Under this assumption, the sampling fractions can overlap for 
a fraction 0i2, with 0i2 satisfying ([T]i. Similar to the point- 
to-point setup of Fig. [T] it is without loss of generality to 
assume the sampling profile is as shown in Fig. |3] The links 



8 



between Encoder 1 and Encoder 2 and between Encoder 2 
and the decoder can support a rate of Ri bits/sample and a 
rate of R2 bits/sample, respectively. As above, the goal is to 
estimate a function T" = /"(S'",S'2 ) at the decoder It is 
observed that if i?i is unbounded, then the scenario reduces 
to the point-to-point system studied in the previous sections. 

Definition 2: A {n, Ri, R2, D, 9i, 62, O12) code for the prob- 
lem of multi-hop lossy computing of two memoryless sources 
with fractional sampling consists of an encoder (Encoder 
1) /i : Sf^ {l,...,2"'"i}; an encoder (Encoder 2) 
/2 : Sf'' X {l,...,2"-«i} ^ {1,...,2"'^2}; and a decoder 
g : {1,...,2"^2} ^ T" such that distortion constraint D is 
satisfied as in (|2]l. It is assumed that encoder /i operates on 
the measurements {Si^i, Si^nOi) and encoder /2 operates 
on the measurements (5'2,„(ei-ei2)+i7 'S'2,n(ei+e2-ei2)) 
well as the index received from Encoder 1. 

The distortion-rate function Z?(i?i, i?2, ^i, ^?2, ^12) and the 
distortion-rate function i?2, 6*1, 6*2) are defined in a 

similar manner as in the point-to-point setup of Section II. 
In the remaining of this section, we focus on the specific case 
in which sources 5*1 and 5*2 are Gaussian and the decoder 
wishes to compute the sum T = Si + 82- Other cases studied 
in Section IV and Section V can also be investigated similarly. 



\S"0, 



Encoder 




Decoder 





Fig. 9. Lossy computing witli side information at the decoder and fractional 
sampling: The encoder measures source Si for a fraction of time 9\, and the 
decoder measures source S2 for a fraction of time 62 and estimates a function 
T" = /"(SJ'.SJ). 



A. Lower Bounds on the Achievable Distortion 

Two lower bounds on the achievable distortion for the 
Gaussian multi-hop lossy computing problem discussed above 
can be derived based on the cut-set arguments |8 1. Specifically, 
the first cut is around Encoder 1 and the second cut is 
around the decoder These two cuts induce the following two 
subproblems of the original problem in Fig. |2] 1) For the cut 
around Encoder 1, the problem is equivalent to point-to-point 
lossy computing with side information, in which the encoder 
and the decoder can only measure a fraction of the n samples 
from the respective source; 2) For the cut around the decoder, 
the problem reduces to that of point-to-point source coding 
problem investigated in Section IV-B, leading to a lower bound 
as given by (|9]l of Proposition |2] with R replaced by i?2. 

In the following, we study the first subproblem identified 
above, namely, the problem of lossy computing with side 
information at the decoder and fractional sampling as shown 
in Fig. |9] To elaborate, the encoder measures a fraction 9i 
of samples from Si and describes it using rate Ri to the 
decoder At the same time, the decoder measures a fraction, 
92, of samples of a correlated source ^2, which overlaps with 
the encoder's measurements for a fraction 9i2 of samples. 



Based on the description received from the encoder and its 
own measurements, the decoder forms the estimate T". 

Proposition 9: For the problem of lossy computing with 
side information at the decoder and fractional sampling, given 
sampling budget (^1,^2) and rate the distortion-rate 
function i)'^^ (_Ri, 0i, ^2) can be obtained as follows. 

1) For p > 0, 



(1 



^ P?{9i- 91^)2 

{l + p)H0l2-&^ 
if Ri < ^ 



92) + 



■logs 



2(1 



where 9 



12 



2) If p < 0, then 



{1 - p^)9l,2-^ - {1 - p^) 
-{l + pf92 + 2{l + p), 



'12 



= < 



if i?i<%log2(i^), 



Rii + R12 < 
and R21 + R22 + R23 < R2- 



(26a) 
(26b) 



9i{i + py ^ 

+ 2p(l + p)0t2-(l+P)'(^l+^2) 

+2(1 + p), otherwise, 

(24) 

is the optimal overlap fraction; 



+2p{l + p)9l^-{l + py-{9i+t 

+2(1 + p), otherwise, 

(25) 

where 6**3 = 6*12. max is the optimal overlap fraction. 
Proof: See Appendix IE] ■ 
Proposition|9]states that, in the setting of Fig.|9] it is optimal 
to have the overlap fraction as small as possible if the two 
sources are positively correlated, and as large as possible if 
they are negatively correlated. This result is consistent and 
follows from similar considerations as Proposition |3] 5] and |5] 

B. Upper Bounds on the Achievable Distortion 

In this subsection, we first propose a specific strategy, thus 
providing an upper bound on the achievable distortion. The 
derived upper bound is then compared to the lower bounds in 
Proposition |2] and Proposition |9] through numerical examples. 

In the proposed strategy, we treat the (6*1 — 6'i2)-fraction 
of samples measured only at Encoder 1, the overlapping 6112- 
fraction measured by both the encoders, and the {92 — 9i2)- 
fraction measured only at Encoder 2 separately in terms of 
encoding and decoding. In particular, on the link between the 
two encoders, which is of rate we assign rate Ru to 
the encoded version of the {9i — 0i2)-fraction of samples and 
rate R12 to the 6'12-fraction. Moreover, on the link between 
Encoder 2 and the decoder, which is of rate R2, we allocate 
rate R21 to forward the encoded version of the (6*1 — ^12)- 
fraction, rate R22 to the 6'12-fraction and rate R23 to the (02 — 
6'i2)-fraction. By definition, we thus have the conditions 
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We specify the source coding strategy used for the different 
fractions of samples and discuss the resulting average distor- 
tions as follows. For the {9i — 6'i2)-fraction measured only 
by Encoder 1, we have available an end-to-end rate equal to 
min(_Rii, i?2i). Encoder 1 thus compresses this fraction of 
samples of Si at rate min(_Rii, _R2i)/(^'i — ^12) bits/source 
sample using a standard indirect rate-distortion optimal code, 

leading to average distortion 1 — p + (1 + p) 2 «i-<'i2 
l|5j. Similarly, for the (6*2 — 6'i2)-fraction of samples measured 
only by Encoder 2, Encoder 2 can employ rate i?23/(fi'2 — ^12) 
using a standard indirect rate-distortion optimal code, lead- 

_ 2i?23 

ing to average distortion 1 — + "2-812 |l5l|. For the 
(1 + 9i2 — Oi — 6'2) -fraction measured by neither node, the 
average distortion at the decoder is equal to the variance 
of source T — Si + S2, namely, 2(1 + p). For the ^12- 
fraction measured by both nodes, the setup at hand reduces to 
the multi-hop source coding problem investigated in ||9] with 
the average link rates over the two links being R12/O12 and 
^^22/^12, respectively. Following the "re-compress" scheme 
proposed in (|9l, the resulting distortion for this fraction 
at the decoder is given by Do{Ri2/9i2, R22/S12), where 
DoiRuR2) - (1 - p')(l - 2-2«^)2-2«i + 2(1 + p)2-2«^ 
for i?i > and R2 > 0. 

Applying the source coding strategy described above, an 
achievable distortion at the decoder is given by summing the 
contributions of the different fractions of samples with the 
appropriate weights as 



-D„b(i?l, i?2, ^'i, ^2 



mm 

; ,-Rll ,R2 



O12D0 



Ri — Rii R2 



2(H2-Hll-H22) 
82-012 



(27) 



+ (l + p)2 {9i - 012)2- + {62-912)2 

+ 2p{l + p)9i2 - (1 + p)HOi + O2) + 2(1 + p), 

where the minimum is taken under the constraints (1) and 1 
Note that in ( |27] |. we have set R12 = i?i — R21 — Rii 
and i?23 = R2 — Rii — R22 without loss of optimality, since 
the two rate bounds in ( |26] l are easily seen to be satisfied with 
equality at an optimal solution. 
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Fig. 10. The upper and lower bounds on the achievable average distortion 
versus link rate R2 for computing T = Si + S2, (Si, S2) jointly Gaussian 
with correlation coefficient p = 0.5. The sampling budget is {01,82) = 
(0.5, 0.75) and the link rate between the encoders is Ri = 0.3. 



that, for the point-to-point case when Ri goes to infinity, if 
rate R2 is small enough, then the optimal overlap fraction 9I2 
equals the largest possible value 9i2,max- Thus, the optimality 
of the choice 9^2 = & 12, min in the multi-hop scenario with the 
given value of Ri is due to the fact that the source Si received 
at Encoder 2 is noisy as a result of the necessary compression 
at Encoder 1. Moreover, since rate R2 is very small, similar 
to Proposition |5] the rates of both links are entirely allocated 
for describing the overlapped fraction of samplesQ As rate R2 
increases, the optimal overlap fraction 6*5^2 gradually increases 
until it reaches the maximum possible value 9i2.max = 0.5. In 
this regime, in addition to the overlapped fraction of samples. 
Encoder 2 also starts describing the non-overlapped fraction of 
samples measured only from source 6*2. Finally, as R2 grows 
beyond 0.16, the non-overlapped fraction of samples measured 
only from source Si also starts being allocated a non-zero rate 
and the optimal overlap fraction 9^2 decreases down to the 
minimum possible value 9i2.min = 0.25. This is consistent 
with Proposition |9] which shows that as R2 goes to infinity, 
the optimal overlap fraction is 9i2.min — 0.25. 



C. Numerical Results 

Fig. [To] and Fig. [TT] plot the achievable distortion 
Dub{Ri, R2,0i,92) and the corresponding optimized overlap 
I2 as a function of rate R2 when Ri = 0.3, 



fraction 

h) 



i,u2) = (0.5,0.75) and p = 0.5. In Fig. \T0\ the two 
lower bounds obtained, D^^^(i?i, i?2, ^i, ^2) from Proposition 
|9]and D^'^^Ri, R2,9i,92) from Proposition |2] are also plotted 
for comparison. We observe that the distortion of the pro- 
posed scheme decreases as R2 increases and gets arbitrarily 
close to lower bound D'^' (i?i, i?2, ^1, 6*2), corresponding to 
the cut around Encoder 1, as R2 gets sufficiently large. 
In contrast, lower bound D'^^^ (i?i, i?2, ^1, ^'2), corresponding 
to the cut around the decoder, is tighter than lower bound 
D'^^\Ri,R2,9i,92) for small rates R2. 

The optimal overlap fraction 9I2 for the achievable rate is 
equal to the minimum possible fraction ^12, mm — 0.25 when 
i?2 is sufficiently small, i.e., R2 < 0.03. It is interesting to 
compare this result with Proposition |5] In fact, the latter entails 



VIII. Conclusions 

In this paper, we considered the problem of lossy compres- 
sion for computing a function of correlated sources. Motivated 
by the fact that acquiring the information necessary for com- 
putation may be costly in sensor networks, we assumed that 
the encoder can only observe a fraction of the samples from 
each source according to a sampling strategy that is subject 
to design. We also investigated the corresponding multi-hop 
problem with two encoders each observing a fraction of one 
of the sources. The results highlight the dependence of the op- 
timal sampling strategy on the function to be computed by the 
decoder, on the source statistics, including the correlation, on 
the link rate and the desired metric for distortion. Interesting 
future work includes investigation of other network scenarios 
and extensions to sources with memory. 

^The fact that equals 9i2,min, despite the fact that only the overlapped 
samples are described, is explained mathematically by the non-convexity of 
function Dq (Ri , -R2 )■ 
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Fig. 11. The optimal overlap fraction that minimizes the achievable 
average distortion versus link rate R2 lor computing T = Si + S2, {Si , S2) 
jointly Gaussian with correlation coefficient p = 0.5. The sampling budget is 
{61, 62) = (0.5, 0.75) and the link rate between the encoders is Ri = 0.3. 



Appendix A 
Proof of Proposition[T] 

Given T = Si, we have the distortion rate functions 
Di{R) = Di2{R) = 2-2« and D2iR) = 1 - + p^2-^^ 
In this case, applying Lemma [H we obtain 



D{R, 01,02,^12) 

- min (01-012)2- 
RuB.2,Ri2>0 



+ {O2 - 612) [l-p^ +p^2 )+ (1 + 012 - 6*1 - 62) 

(28) 

2B.2 



= min 6*12 ~i + (0, - 012)0^2 ^^^^ 

0<R2<R 

+ i-ei-p\e2-ei2), 



(29) 

where the minimization in (|28] l is under the constraint i?i + 
i?2 + ^12 < R- Note that the optimization in ( |28] ) is 
equivalent to that in ( |29l ). since in any optimal solution, we 
have R12/O12 = Ri/{Oi — 612) by the convexity of function 
2-2r fQj. r > 0, and the condition Ri + R2 + R12 < R 
must be met with equality. It can be easily seen that function 
D{R, 61,62, 612) above is monotonically non-decreasing with 
respect to 6*12. Therefore, the optimal overlap is the minimum 
possible, which equals 6^2 — ^'12, min- Moreover, the optimal 
rate R2 that minimizes ( |29] l can be obtained using standard 
Lagrangian methods similar to [3| as: 



R 



61, 1 
2 p^ 



(30) 



oi -\- 62 ~ 612 

With the so obtained R2, the results in Proposition [T] follows 
immediately. 



that, during the 6*1 2 -fraction, T can be computed at the encoder 
and described to the decoder losslessly with a rate equal to the 
entropy of T, h{{l - p)/2). During the {61 - 6'i2)-fraction, 
only source 5*1 is observed and can be described to the decoder 
losslessly with a rate h{l/2) = 1. Based on source Si, the best 
estimate at the decoder is as follows: T = if 6*1 = 0, and 
T = 1 if 5i = 1, leading to average Hamming distortion 
p/2. Similarly, during the {62 — 6'i2)-fraction, the average 
Hamming distortion is alsop/2. During the {I + 612 — 61—62)- 
fraction, neither source Si nor source ^2 is observed. Since 
T is Bernoulli distributed with (1 — p)/2, the best estimate 
is given by t = 0, leading to average Hamming distortion 
(1 — p)/2. Therefore, we have Di.mm — D2.min — p/2 and 
Dmax = (1 ^p)/2- Substituting into (|7]l of Lemma|2] we have 



^2) = 



1-p 



1 



] ^^12: 

(31) 



where 6*12 = 6'i2,mi„ if p < 1/3, and 6*12 = 6*12, max if P > 
1/3. Finally, from the discussion above, it follows that, for 
any R > R,mn{6i,62) ^ 61 + 62 - (2 - /i((l - p)/2)) ^1*2, 
distortion D„iin{6i,62) can be achieved at the decoder 



Appendix C 
Proof of Lemma|4] 

In the case of indirect description of T based on only source 
Si, the indirect rate-distortion function is given by Ri{D) = 



■^p(t>i): Ed(T,T)<D 



I{Si;T) \2\. Let p{t = l|si = 0) 



and p{i = l|si = 1) = y, where < a; < 1 and < y < 1. 
Note that if we select x = y = Q, i.e., T = with probability 
1, the average distortion L) = (l — p)/2is achievable at the 
decoder. Thus, for D > [1 — p)/2, we have Ri{D) = 0. 
Moreover, from the proof of Proposition |7] it follows that D > 
p/2 must hold. For the nontrivial case p/2 < D < (1— p)/2, 
the expected distortion constraint can be written as 



i?(d(T,r)) = - 



-(i-p) + fp 



x + {2p-l)y+l-p 



< D, 



and the mutual information I{Si;T) can be written as 



IiSi;f)^H{T)-H{T\Si)^h 



x + y 



(32) 



1 



-h{x) - -h{y) 
(33) 

For any given y, considering the monotonicity of (l33T l with 
respect to x for < x < 2c?— (1 —p) + {l — 2p)y, we can easily 
show that ( |33] | is minimized at a; = 2(i— (1— p)-|-(l— 2p)y, i.e., 
(|32] | is met with equality. Therefore, for p/2 < D < (1— p)/2, 
we obtain Ri{D) as in ( fTSl l. 



Appendix B 
Proof of Proposition!?] 

For any given sampling profile {61,62,612), in order to 
minimize the distortion with respect to R, we can take R 
to be arbitrarily large (in fact, given the binary alphabets, 
R — 1 suffices). With no rate limitations, it is easy to see 



Appendix D 
Proof of Proposition[8] 

If we set T = at the decoder, the resulting Hamming 
distortion is (1 — p)/2. Hence, for D > (\ — p)/2, zero 
rate is required for description, i.e., R{D ,61,62) = 0. For 
Dmin{6i,62) < D < for any given sampling profile 



11 



(6*1, 6*2, ^12), we can use Lemma [T] by setting Di,„iin = 

D2,min = p/2, Di2^rnin = and Dmax = (1 " P)/2. Due 

to the convexity of Ri{D), it is optimal to have Di/{6i — 
O12) ~ ^2/(^*2 — (^12) in any optimal solution. Moreover, with 
Dmin{0i,02) < D < for optimality, (|6]l must be met 

with equality, i.e., 

D,+D2+D,2+ ^'^'^^-\-'^^^'-'^ =D, (34) 

and Di, D12 and D2 must be such that Di/ {61—612), D12/612 
and 02/(62 — 612) are all less than or equal to Dmax = 
(1 ~p)/2. If we let D3 ^ Di+ D2, then D3 satisfies 

2 2 ■ ^^^^ 

Finally, taking the minimum of R{D,6i,62,6i2) over all 612 
satisfying we obtain R{D, 61,62) as in the proposition for 

2 



Dm^niel,62)<D<^^. 



Appendix E 
Proof of Proposition^ 
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For any given sampling profile (6'i, ^2, ^'12), we denote by 
Rii and R12, the rates used by the encoder in Fig. |9] to 
describe the non-overlapping {61 — 0i2)-fraction of samples 
and the overlapping 0i2-fraction of samples measured from 
Si, respectively. During the (?i2-fraction, using rate R12/612, 
one can achieve average distortion (1 — p2^2^2fl'i2/6'i2 ]jy )jjg 
Wyner-Ziv theorem [10|. During the {61 — 0i2)-fraction, only 
source Si is observed and described to the decoder using 
rate Rii/{6i — 612) and the resulting average distortion is 
given by 2(1 + - p2 ^_ ^22-2i?,ii/(ei-ei2))^ where p is as 

defined in Section IV. During the (^2— ^'i2)-fraction, since only 
^2 is observed perfectly at the decoder, the resulting average 
distortion can be easily seen to be 1 — p^. Finally, during the 
(1 + 012 — 6i~ 02 ) -fraction, with neither source 5*1 nor source 
S2 observed, the average distortion at the decoder is equal to 
the variance of T, namely, 2(1 + p). From the independence of 
samples measured from the different fractions of samples, the 
minimum achievable distortion for sampling budget {61,62) 
and rate Ri can be obtained as 

D^^\Ri,6i,62) ^ min {1 - p'^)6i2'2~^ 

O12 ,Rii ,Ri2 

+ (1 + p)\ei - 612)2-^^ + 2p{i + p)6i2 

-{l + pf {61+62) + 2{\ + p), (36) 

where the constraint on 0i2 is as in (1) and the constraint 
on Rii,Ri2 is given by i?ii + i?i2 < Ri- The minimum 
achievable distortion in the proposition is obtained by solving 
the optimization problem ( |36] |. Specifically, for p > 0, we 
can show that it is optimal to have 6^2 = 0i2,miTi by simply 
considering the monotonicity of function 0i, 02, ^12) 

with respect to 0i2. Similarly, we can show that for p < 0, 
it is optimal to have 6^2 = ^12. ?riai- Moreover, the optimal 
rate Rl^ and RI2 can be obtained using standard Lagrangian 
methods similar to |I3|. Details are omitted. 



